WIP: harbor backend v2 - #272
Draft
mathewjhan wants to merge 29 commits into
Draft
Conversation
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Why
How to Test
Checklist
[module] type: descriptionformatenhancement,bug,breaking)ruff check .andruff format --check .passpyright osmosis_ai/passespytestpasses (new tests added if applicable)Summary by cubic
Add
HarborBackendV2to run Harbor tasks with pure task images via bundled workflows or native Harbor agents, plus server-side admission control and rollout cancellation for safer queuing.New Features
HarborBackendV2: runs bundled workflows or native agents (terminus-2,mini-swe-agent,oracle); supports template/dataset modes; per‑phase timing with failure diagnostics; redacted diagnostics exposed viaextra_fieldson samples.osmosis_ai.packaging): builds a wheel from apyproject.tomlharness, injects agent/grader console scripts, caches by content hash in the user cache dir viaplatformdirs, and exposesinspect_bundle/project_dir_for.ContainerInput/ContainerResultplus in‑containerrunner.agent_main/runner.grader_main; extracts messages from trajectories, writes reward files for Harbor, and addsAgentWorkflowOutput.HarborTask+TaskModecopy per rollout, writeinstruction.md, patch Dockerfiles to install SDK deps into an isolated venv (/opt/osmosis/venv) bootstrapped withuv(fallback topip), and generatetests/test.shto install a grader when needed.HarborBackend: content‑addressed image tags to reuse builds; new exports inosmosis_ai.rollout.backend.harbor.__init__.has_capacity()(returns 429 +Retry-Afterwhen full) and/rollout/cancel(ids/prefix/all) wired throughcancel_rollouts()in the backend.benchmarks/container_lifecycleto measure cold/warm throughput.Migration
HarborBackendV2withtasks_dir; choose a native agent by name or provide a workflow harness (code_diror a prebuiltbundle).pyproject.toml; the bundler creates<package>-agent/<package>-gradescripts automatically./opt/osmosis/venv; images only need Python. For dataset runs, settask_mode="dataset"and includeharbor_task_idin request metadata.Retry-After) and may call/rollout/cancelto stop queued/running rollouts.Written for commit c42cf18. Summary will update on new commits.